Analyzing Random Permutations for Cyclic Coordinate Descent
نویسندگان
چکیده
We consider coordinate descent methods on convex quadratic problems, in which exact line searches are performed at each iteration. (This algorithm is identical to Gauss-Seidel on the equivalent symmetric positive definite linear system.) We describe a class of convex quadratic problems for which the random-permutations version of cyclic coordinate descent (RPCD) outperforms the standard cyclic coordinate descent (CCD) approach. A convergence analysis is developed to support the computational observations. Coordinate descent, Gauss-Seidel, randomization, permutations
منابع مشابه
Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems
The iteration complexity of the block-coordinate descent (BCD) type algorithm has been under extensive investigation. It was recently shown that for convex problems the classical cyclic BCGD (block coordinate gradient descent) achieves an O(1/r) complexity (r is the number of passes of all blocks). However, such bounds are at least linearly depend on K (the number of variable blocks), and are a...
متن کاملDescents of λ-unimodal cyclic permutations
We prove an identity conjectured by Adin and Roichman involving the descent set of λ-unimodal cyclic permutations. These permutations appear in the character formulas for certain representations of the symmetric group and these formulas are usually proven algebraically. Here, we give a combinatorial proof for one such formula and discuss the consequences for the distribution of the descent set ...
متن کاملDescent sets of cyclic permutations
We present a bijection between cyclic permutations of {1, 2, . . . , n+1} and permutations of {1, 2, . . . , n} that preserves the descent set of the first n entries and the set of weak excedances. This non-trivial bijection involves a Foata-like transformation on the cyclic notation of the permutation, followed by certain conjugations. We also give an alternate derivation of the consequent res...
متن کاملOn the Nonasymptotic Convergence of Cyclic Coordinate Descent Methods
Cyclic coordinate descent is a classic optimization method that has witnessed a resurgence of interest in Signal Processing, Statistics and Machine Learning. Reasons for this renewed interest include the simplicity, speed, and stability of the method as well as its competitive performance on `1 regularized smooth optimization problems. Surprisingly, very little is known about its non-asymptotic...
متن کاملOn the Finite Time Convergence of Cyclic Coordinate Descent Methods
Cyclic coordinate descent is a classic optimization method that has witnessed a resurgence of interest in machine learning. Reasons for this include its simplicity, speed and stability, as well as its competitive performance on l1 regularized smooth optimization problems. Surprisingly, very little is known about its finite time convergence behavior on these problems. Most existing results eithe...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2017